Skip to content

[java] Add linux-x64 implementation of in process Copilot CLI - #2301

Open
edburns wants to merge 1 commit into
mainfrom
edburns/1917-java-embed-rust-cli-runtime-dd-3042873-seeking-review-03
Open

[java] Add linux-x64 implementation of in process Copilot CLI#2301
edburns wants to merge 1 commit into
mainfrom
edburns/1917-java-embed-rust-cli-runtime-dd-3042873-seeking-review-03

Conversation

@edburns

@edburns edburns commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Supercedes #2295 .

This PR is the roll up of the agentic work done in the subtasks of #2166 . At each step of those subtasks, the CI was clean and all reviews were applied as appropriate.

PR 2295 — Reviewer's guide: In-process FFI runtime for the Java SDK

TL;DR

This PR does for the Java SDK what #1901 did for .NET and #1915 did for Rust: it adds an in-process connection mode that loads the Copilot runtime (runtime.node cdylib) as a native library via JNA, eliminating the need for a separate CLI child process. Currently scoped to linux-x64 only; the entire in-process API surface is marked @CopilotExperimental.

The PR also restructures the Java Maven project from a single module into a multi-module reactor to support publishing the native runtime binaries as separate classifier JARs alongside the existing SDK JAR.


What's in the native binary, where does it come from, and how is it loaded?

The binary: runtime.node

Despite the .node extension (a napi-rs naming convention), runtime.node is an ordinary platform-specific shared library (.so on Linux). It is a Rust cdylib produced by the src/runtime crate in github/copilot-agent-runtime. It exposes two front doors:

  • napi front door — loaded by Node.js as a native addon (existing CLI path).
  • C ABI front door — 5 extern "C" lifecycle/transport entry points callable by any language via FFI without Node.js.

The 5 C ABI entry points are:

Entry point Purpose
copilot_runtime_host_start Start the runtime host. Blocks up to ~30s while the worker boots. Returns a server handle (0 = failure).
copilot_runtime_host_shutdown Shut down a runtime host by server handle.
copilot_runtime_connection_open Open a bidirectional connection; registers an on_outbound callback for runtime→SDK data delivery.
copilot_runtime_connection_write Write a JSON-RPC frame from the SDK into the runtime.
copilot_runtime_connection_close Close a connection.

All JSON-RPC methods travel as data through this fixed 5-function transport; the export surface never changes as the method set grows.

Where it comes from (build-time)

The copilot-native Maven module's build fetches the binary from npm during generate-resources:

  1. fetch-native.mjs reads the pinned version and SHA-512 integrity hash for @github/copilot-linux-x64 from nodejs/package-lock.json.
  2. Runs npm pack to download the exact tarball, verifies it against the integrity hash.
  3. Extracts runtime.node and the copilot CLI executable into a staging directory.
  4. maven-jar-plugin packages them into a classifier JAR (copilot-sdk-java-runtime-<version>-linux-x64.jar) with the layout native/linux-x64/runtime.node.

How it's loaded (runtime)

  1. PlatformDetector (303 lines) determines the classifier using os.name, os.arch, and on Linux, ELF PT_INTERP parsing to distinguish glibc vs musl — no subprocesses, no heuristics.
  2. NativeRuntimeLoader (466 lines) resolves the binary in this order:
    • COPILOT_CLI_PATH env var → checks for runtime.node alongside the CLI.
    • Classpath resource native/<classifier>/runtime.node → extracts atomically to ~/.copilot/runtime-cache/<version>/<classifier>/runtime.node.
    • Falls back to runtime.node alongside the bundled copilot executable.
  3. JnaNativeBinding (253 lines) loads the library by absolute path via JNA and maps each C ABI export. Enforces a one-library-per-process invariant (library handle is static, never unloaded). Duplicate loads from the same path are silently accepted; different paths are rejected.
  4. FfiRuntimeHost (349 lines) orchestrates the lifecycle: starts the host, opens a connection, bridges the bidirectional JSON-RPC transport. The on_outbound callback (invoked by native threads) feeds received data into a QueueInputStream that the SDK's existing JsonRpcClient reads from.

Structural changes

Multi-module Maven reactor

The single-module java/pom.xml is now a parent POM (pom packaging) with two submodules:

Module Artifact ID Purpose
java/pom.xml copilot-sdk-java-parent Reactor parent. Not published to Maven Central (maven.deploy.skip=true). Holds the release profile (GPG signing) inherited by all submodules.
java/sdk/ copilot-sdk-java The existing SDK JAR (~1.5 MB). All existing source moved here from java/src/java/sdk/src/.
java/copilot-native/ copilot-sdk-java-runtime Native runtime module. Produces classifier JARs (currently linux-x64 only, ~20-26 MB).

Consumer dependency declaration

<dependencies>
    <!-- Pure-Java SDK (~1.5 MB) -->
    <dependency>
        <groupId>com.github</groupId>
        <artifactId>copilot-sdk-java</artifactId>
        <version>${copilot.version}</version>
    </dependency>
    <!-- Native runtime for linux-x64 (~20-26 MB) — needed only for in-process mode -->
    <dependency>
        <groupId>com.github</groupId>
        <artifactId>copilot-sdk-java-runtime</artifactId>
        <version>${copilot.version}</version>
        <classifier>linux-x64</classifier>
    </dependency>
</dependencies>

Consumer usage

CopilotClientOptions options = new CopilotClientOptions()
    .setConnection(RuntimeConnection.forInProcess());

CopilotClient client = new CopilotClient(options);
client.start().get();

New public API surface (all @CopilotExperimental)

Type Description
RuntimeConnection (sealed class) Base type for connection configuration. Factory methods: forStdio(), forTcp(), forUri(String), forInProcess().
StdioRuntimeConnection Spawns a runtime child process, communicates over stdin/stdout (the default).
TcpRuntimeConnection Spawns a runtime child process listening on a TCP socket.
UriRuntimeConnection Connects to an already-running runtime at a URL.
InProcessRuntimeConnection Loads the native library in-process — no child process spawned.
CopilotClientOptions.setConnection() / getConnection() Entry point for selecting a connection type.

The RuntimeConnection API replaces the previous pattern of setting cliUrl, cliPath, useStdio, port, and tcpConnectionToken individually. When a RuntimeConnection is set, it takes precedence; conflicting legacy options cause IllegalArgumentException.


New internal packages

com.github.copilot.ffi (9 classes, ~1,752 lines)

Class Lines Role
FfiRuntimeHost 349 Lifecycle manager: start host → open connection → bridge I/O → shutdown.
JnaNativeBinding 253 JNA bindings for the 5 C ABI exports. Static library handle, one-per-process guard.
NativeBinding 131 Abstract contract for native operations (enables testing without real native library).
NativeRuntimeLoader 466 Locates runtime.node: env var → classpath → cache. Atomic extraction with file locking.
PlatformDetector 303 Determines platform classifier. ELF PT_INTERP parsing for glibc/musl detection on Linux.
QueueInputStream 119 Thread-safe bridge: native callback thread writes → SDK reader thread reads.
FfiOutputStream 63 Writes JSON-RPC frames from the SDK into the native runtime via connection_write.
OutboundCallback 46 JNA callback implementation for on_outbound.
ReaderThreadFactory 22 Named daemon thread factory for the reader executor.

Tests for FFI (6 files, ~2,054 lines)

Test class What it covers
FfiRuntimeHostTest Lifecycle, error handling, concurrent shutdown, callback drain.
JnaNativeBindingTest Load guard, duplicate-path acceptance, different-path rejection, active callback tracking.
NativeRuntimeLoaderTest Resolution order, atomic extraction, COPILOT_CLI_PATH override, cache reuse.
PlatformDetectorTest All 8 platform classifiers, ELF parsing, edge cases.
QueueInputStreamTest Thread-safe read/write, close semantics.
InProcessTransportIT End-to-end integration test using the replay proxy with in-process transport.

CI/workflow changes

  • New job java-sdk-inprocess in java-sdk-tests.yml: runs mvn clean verify -Pinprocess on ubuntu-latest (linux-x64). Uses continue-on-error: true while experimental.
  • Path updates in existing jobs: java/target/java/sdk/target/ for surefire/failsafe reports and coverage data.
  • JDK 17 cross-test: added -pl sdk to restrict to the SDK module (the native module requires JDK 25 build tools).
  • Codegen workflows: adjusted working directories for the java/sdk/ module layout.

✅ Note that the existing java publishing jobs will continue to work as currently written.


Key design decisions (from ADR-007)

  1. JNA over Panama FFM: JNA supports the Java 17 baseline with zero consumer configuration. Panama FFM requires Java 22+ and --enable-native-access flags. Performance difference is irrelevant (JSON-RPC I/O dominates).

  2. Per-platform classifier JARs over monolithic JAR: A monolithic JAR with all 6 common platforms would be ~132 MB. Classifier JARs let consumers pull only their target platform (~20-26 MB each). An uber-JAR can be assembled via maven-assembly-plugin if needed.

  3. Library-never-unloads pattern: The loaded native library is held in a static field and never released. Native worker threads outlive any individual FfiRuntimeHost instance; unloading would crash.

  4. One library per process: Enforced by a process-wide guard, consistent with Rust, .NET, Go, and Python SDK implementations.


Diff statistics

  • 107 commits, 1,582 files changed (mostly renames from java/src/java/sdk/src/)
  • ~6,624 insertions, ~825 deletions
  • New production code: ~2,117 lines (FFI + RuntimeConnection API)
  • New test code: ~2,054 lines
  • New build infrastructure: copilot-native/pom.xml (214 lines), fetch-native.mjs (114 lines)

Recommended review order

  1. ADR-007: java/docs/adr/adr-007-native-bundling-strategy.md — context, options considered, decision rationale.
  2. RuntimeConnection API: rpc/RuntimeConnection.java, rpc/InProcessRuntimeConnection.java, and rpc/CopilotClientOptions.java (the setConnection/getConnection methods).
  3. FFI bridge (bottom-up): NativeBinding.javaJnaNativeBinding.javaFfiRuntimeHost.javaNativeRuntimeLoader.javaPlatformDetector.java.
  4. Native module build: copilot-native/pom.xml and copilot-native/scripts/fetch-native.mjs.
  5. Multi-module restructure: java/pom.xml (parent) and java/sdk/pom.xml (child).
  6. CI: .github/workflows/java-sdk-tests.yml (new inprocess job, path updates).
  7. Tests: ffi/ test package and e2e/InProcessTransportIT.java.

Implementation details.

Implemented agentically using https://aka.ms/coreai/shepherd-task/slides .

Squashed from PR #2295 (branch edburns/…-review-02).
Includes Java multi-module Maven restructure, copilot-native
submodule for bundling the Rust CLI runtime, codegen updates,
and related workflow changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 90cbda40-cda3-4ecd-b381-9f9ba0573d0a
@edburns
edburns requested a review from a team as a code owner August 9, 2026 20:54
Copilot AI balanced review requested due to automatic review settings August 9, 2026 20:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Cross-SDK Consistency Review ✅

This PR adds the in-process FFI runtime connection to the Java SDK, bringing it to parity with all other SDK implementations.

Feature parity check

SDK In-process support API
Node.js ✅ (existing) RuntimeConnection.forInProcess()InProcessRuntimeConnection
Python ✅ (existing) RuntimeConnection.for_inprocess()InProcessRuntimeConnection
Go ✅ (existing) InProcessConnection{} struct literal
.NET ✅ (existing, PR #1901) RuntimeConnection.ForInProcess()InProcessRuntimeConnection
Rust ✅ (existing, PR #1915) Transport::InProcess enum variant
Java this PR RuntimeConnection.forInProcess()InProcessRuntimeConnection

API naming consistency

The Java implementation follows the expected language idioms:

  • Factory method RuntimeConnection.forInProcess() aligns with Node.js (forInProcess) and .NET (ForInProcess)
  • Sealed class hierarchy (RuntimeConnectionStdioRuntimeConnection, TcpRuntimeConnection, UriRuntimeConnection, InProcessRuntimeConnection) mirrors Node.js and .NET
  • Java camelCase for methods and PascalCase for classes is consistent with the SDK's existing conventions

Conclusion

No cross-SDK consistency issues found. This PR completes the in-process FFI feature across all six SDK languages.

Generated by SDK Consistency Review Agent for #2301 · sonnet46 31.2 AIC · ⌖ 4.07 AIC · ⊞ 6.6K ·

@roji roji left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@edburns here are some comments from my review agent, hope these make sense. Happy to take another human look afterwards!


Requesting changes. The overall transport architecture broadly aligns with the other SDKs, and the large file count is mostly explainable: 1,522 of 1,596 files are byte-identical moves into java/sdk/. The module split is reasonable, but the published dependency graph, native ABI/lifecycle, and release validation still have blocking issues.

GitHub cannot attach inline review comments to unchanged files, so these relocation omissions are called out here:

  • .github/workflows/java-publish-maven.yml:204,207 still references java/jbang-example.java, so release preparation will fail after the move to java/sdk/jbang-example.java.
  • scripts/docs-validation/validate.ts:388-394 searches the parent POM for artifact copilot-sdk-java; it now falls back to 1.0.0-SNAPSHOT instead of validating the reactor's 1.0.11-preview.0-SNAPSHOT artifact.
  • .github/actions/java-test-report/action.yml:7,11,15 still searches java/target/**; current CI logs report that no test reports were found even though results are under java/sdk/target/**.
  • .github/workflows/java-smoke-test.yml:66,139 still points to the pre-move prompt path.

Please address the inline findings and these unchanged-file omissions before merging.

Comment thread java/pom.xml

<readonly-copilot-sdk-ref-impl-version-from-lastmerge-file-updated-by-reference-impl-sync>^1.0.79-6</readonly-copilot-sdk-ref-impl-version-from-lastmerge-file-updated-by-reference-impl-sync>
<!-- The parent POM is not published to Maven Central. -->
<maven.deploy.skip>true</maven.deploy.skip>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker: Both published child POMs inherit from copilot-sdk-java-parent, but this prevents that parent from being deployed and there is no flattening step. Maven consumers resolving copilot-sdk-java or copilot-sdk-java-runtime will then fail to resolve their parent POM. Please publish the parent or deploy flattened child POMs.

Comment thread java/pom.xml
-->
<readonly-copilot-sdk-ref-impl-version-from-lastmerge-file-updated-by-reference-impl-sync>^1.0.79-9</readonly-copilot-sdk-ref-impl-version-from-lastmerge-file-updated-by-reference-impl-sync>

<readonly-copilot-sdk-ref-impl-version-from-lastmerge-file-updated-by-reference-impl-sync>^1.0.79-6</readonly-copilot-sdk-ref-impl-version-from-lastmerge-file-updated-by-reference-impl-sync>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High: This pin is being downgraded from ^1.0.79-9 to ^1.0.79-6, and the regenerated public API consequently loses current types and fields such as SandboxConfigAuth, factory-agent options, SubagentCompletedEvent.cancelled, and paged listRuns. This is unrelated to the FFI work and rolls main backwards. Please restore the current pin and regenerate.

Comment thread java/README.md
<!-- Native runtime for linux-x64 (~20-26 MB) -->
<dependency>
<groupId>com.github</groupId>
<artifactId>copilot-sdk-java-runtime</artifactId>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker: These two artifacts are not sufficient for in-process mode. The SDK declares JNA as optional, and the runtime artifact does not depend on JNA, so Maven will not bring it transitively. Users following this snippet will hit NoClassDefFoundError when selecting the in-process transport. Please make the runtime artifact bring JNA transitively or document an explicit third dependency.

try {
return lib.copilot_runtime_connection_close(connectionId) != 0;
} finally {
trackedCallbacks.remove(connectionId);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High: Removing the wrapper in finally releases its only GC root even when native connection_close fails or throws. Native code may still retain and invoke that function pointer, which can crash the JVM after JNA collects the callback. Keep the wrapper rooted through callback draining and host shutdown, and only release it when native ownership has definitely ended.

</plugin>
<!--
Required by Maven Central: sources and javadoc artifacts. This
module has no Java sources, so both produce empty archives.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High: This claim does not match the release output. Building this module with -Prelease emits the main and sources JARs but logs No Javadoc in project. Archive not created; Maven Central requires a javadoc JAR for every non-POM artifact. Please attach an explicit placeholder javadoc JAR for this source-less module.

* @param len
* byte length of the buffer pointed to by {@code data}
*/
void invoke(Pointer userData, Pointer data, int len);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: The C ABI declares this length as size_t, but Java int is always 32-bit and therefore mismatches the 64-bit ABI. The other SDKs use c_size_t, size_t, nuint, or usize. Please use a JNA IntegerType sized with Native.SIZE_T_SIZE here and for every native length parameter.

this.receiveStream = Objects.requireNonNull(receiveStream, "receiveStream must not be null");
this.sendStream = new FfiOutputStream(this.nativeBinding, this.connectionId, this.closing, this.operationLock);
this.libraryPath = libraryPath;
Native.setCallbackExceptionHandler((Callback callback, Throwable throwable) -> LOG.log(Level.WARNING,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: This replaces JNA's process-wide callback exception handler for the entire hosting application and never restores it, changing the behavior of unrelated JNA callbacks. The local outbound callback already catches and logs failures, so please avoid this global mutation or preserve and restore the prior handler.

name: "Java SDK InProcess Tests"
if: github.event.repository.fork == false
runs-on: ubuntu-latest
continue-on-error: true

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: Making the only real in-process test job continue-on-error means FFI regressions can never block this PR or later merges. Since this change introduces the transport, this job should be required once it is added.

Comment thread java/README.md
You can run the SDK without setting up a full Java project, by using [JBang](https://www.jbang.dev/).

See the full source of [`jbang-example.java`](jbang-example.java) for a complete example with more features like session idle handling and usage info events.
See the full source of [`jbang-example.java`](sdk/jbang-example.java) for a complete example with more features like session idle handling and usage info events.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: This updates the source link after the move, but the runnable JBang URL immediately below still points to java/jbang-example.java. The release workflow also retains that old path. Please update both remaining references to java/sdk/jbang-example.java.

Comment thread java/README.md
- `.overridesBuiltInTool(boolean)` — shadow built-in tools

For design context and decision rationale, see [ADR-006](docs/adr/adr-006-tool-definition-inline.md).
For design context and decision rationale, see [ADR-006](sdk/docs/adr/adr-006-tool-definition-inline.md).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: The ADRs remain under java/docs/adr/, so sdk/docs/adr/... does not exist. This link should remain docs/adr/adr-006-tool-definition-inline.md; the ADR-004 link later in the README needs the same correction.

@roji roji left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here are a few more comments.

Another thing I noticed is that while all other language SDKs automatically download the correct platform package with the native binary, the current approach in this PR requires users to manually take a dependency on e.g. the linux-x64 package, in addition to the platform-agnostic SDK package.

I don't know anything about how this kind of thing works with Java/Maven; is it impossible/not "the right way" to offer something that does this automatically (as all the other SDKs do)? Or maybe you're planning to look at that separately in a later PR (obviously completely fine too). Just raising the question.

contents: read

jobs:
java-sdk-inprocess:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to integrate this in the regular job as a matrix ([default, inprocess]) rather than having a completely separate leg for it?

Additional classifiers are added in a later phase, each with its own
fetch execution and maven-jar-plugin execution.
-->
<copilot.native.classifier>linux-x64</copilot.native.classifier>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean we don't support any other platform (linux-arm64, Windows/Mac)? Of course totally fine if you want to do this incrementally in separate PRs, just pointing it out.

String classifier = PlatformDetector.detectClassifier();
String version = readVersion(loader);
Path cacheBase = defaultCacheBase();
return resolve(null, findRuntimeOnPath(), cacheBase, loader, classifier, version);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review comment:

Could we avoid automatically falling back to an arbitrary copilot found on PATH for in-process mode? The embedded native ABI must stay compatible with the CLI/runtime pair, while a PATH installation can be any version and may produce difficult-to-diagnose ABI skew. The other SDKs use bundled/pinned assets or an explicit COPILOT_CLI_PATH; requiring one of those here would keep Java aligned and make runtime selection deterministic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants